<html>
<head><meta charset="utf-8"><title>reproducible memory usage? · t-compiler/performance · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/index.html">t-compiler/performance</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html">reproducible memory usage?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="232838174"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232838174" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232838174">(Apr 01 2021 at 23:13)</a>:</h4>
<p>Hey <span class="user-mention" data-user-id="119009">@eddyb</span>, would the <em>extensive</em> work you did on reproducible instruction counts also make it possible to get reproducible memory usage, to make statistics like max-rss more useful and more usable? Right now they fluctuate too much to detect small changes in memory usage.</p>



<a name="232871133"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232871133" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232871133">(Apr 02 2021 at 07:46)</a>:</h4>
<p>No, I don't think so. The thing is that the memory usage should be deterministic it is very sensitive to the order, size and amount of allocations. To get more stable memory usage measurements during benchmarking, you need to hook into the allocator to get the amount of actually allocated memory.</p>



<a name="232876393"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232876393" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232876393">(Apr 02 2021 at 08:53)</a>:</h4>
<p>I was under the impression that eddyb's work on deterministic instruction counts ended up having to deal with the same issue in order to get reproducible results.</p>



<a name="232876611"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232876611" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232876611">(Apr 02 2021 at 08:56)</a>:</h4>
<p>Jemalloc has a background thread that frees memory to the OS after a certain period. I believe this period is long enough to not trigger on most benchmarks though.</p>



<a name="232911821"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232911821" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232911821">(Apr 02 2021 at 15:24)</a>:</h4>
<p>I wonder how zig-like allocator scheme fares in reproducibility here. But imho if we are looking to measure memory the easiest way to reproducible results is to record the current allocated size in the program itself potentially by wrapping an existing allocator.</p>



<a name="232911923"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232911923" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232911923">(Apr 02 2021 at 15:25)</a>:</h4>
<p>Current allocated size won't be affected by the implementation of allocator as much that way.</p>



<a name="232912146"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232912146" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232912146">(Apr 02 2021 at 15:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="123586">nagisa</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/reproducible.20memory.20usage.3F/near/232911821">said</a>:</p>
<blockquote>
<p>I wonder how zig-like allocator scheme fares in reproducibility here. But imho if we are looking to measure memory the easiest way to reproducible results is to record the current allocated size in the program itself potentially by wrapping an existing allocator.</p>
</blockquote>
<p>Which kind of wrapping do you mean? doing something with LD_PRELOAD to intercept <code>malloc</code>? Or overriding the Rust global allocator? (I would love to say “the latter”, but of course that doesn’t fully address the case I care most about, namely <code>rustc</code> itself.)</p>



<a name="232912580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232912580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232912580">(Apr 02 2021 at 15:31)</a>:</h4>
<p>Latter. I did employ that kind of strategy in embedded firmware to a great success. But I also had a chance to actually make all of the users actually go through the API that employed such tracking.</p>



<a name="232912646"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232912646" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232912646">(Apr 02 2021 at 15:32)</a>:</h4>
<p><code>massif</code> comes to mind as an external tool that would do the ld_preload kind of stuff for you.</p>



<a name="232912654"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232912654" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232912654">(Apr 02 2021 at 15:32)</a>:</h4>
<p>Would a dumb bump allocator without ASLR or guard pages and disabling free() be feasible for small benchmarks or would that still exhaust memory too quickly? Then RSS should be the sum of all allocations instead of the peak live set.</p>



<a name="232912784"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232912784" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232912784">(Apr 02 2021 at 15:33)</a>:</h4>
<p>I wonder if any of the jemalloc's diagnostic tooling/APIs expose anything like max allocated/total allocated too.</p>



<a name="232913152"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232913152" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232913152">(Apr 02 2021 at 15:36)</a>:</h4>
<p>yeah jemalloc collects some information it seems:</p>
<blockquote>
<p>stats.allocated (size_t) r- [--enable-stats]</p>
<p>Total number of bytes allocated by the application. </p>
<p>stats.active (size_t) r- [--enable-stats]</p>
<p>Total number of bytes in active pages allocated by the application. This is a multiple of the page size, and greater than or equal to stats.allocated. This does not include stats.arenas.&lt;i&gt;.pdirty, stats.arenas.&lt;i&gt;.pmuzzy, nor pages entirely devoted to allocator metadata.</p>
</blockquote>
<p>from <a href="https://nxmnpg.lemoda.net/3/malloc_stats_print">https://nxmnpg.lemoda.net/3/malloc_stats_print</a></p>



<a name="232913238"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232913238" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232913238">(Apr 02 2021 at 15:37)</a>:</h4>
<p>the question is if its possible to easily obtain these numbers more directly than printing them out and then parsing them back.</p>



<a name="232913851"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232913851" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232913851">(Apr 02 2021 at 15:42)</a>:</h4>
<p>(well at least one can ask the stats to be printed in json format… <span aria-label="lol" class="emoji emoji-1f606" role="img" title="lol">:lol:</span> )</p>



<a name="232914013"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914013" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914013">(Apr 02 2021 at 15:44)</a>:</h4>
<p>but: sort of sad that it really does only do a print out. Like, why not an interface with a callback that takes a <code>(key: *const char, val: *const u64)</code> pair?</p>



<a name="232914113"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914113" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914113">(Apr 02 2021 at 15:45)</a>:</h4>
<p>(security concerns?)</p>



<a name="232914157"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914157" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914157">(Apr 02 2021 at 15:45)</a>:</h4>
<p>Modifiability maybe? So that people don't depend on what's effectively an internal counter most likely.</p>



<a name="232914544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914544">(Apr 02 2021 at 15:49)</a>:</h4>
<p>I think there are such apis</p>



<a name="232914578"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914578" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914578">(Apr 02 2021 at 15:49)</a>:</h4>
<p><a href="https://docs.rs/jemalloc-ctl/0.3.3/jemalloc_ctl/stats/struct.allocated_mib.html">https://docs.rs/jemalloc-ctl/0.3.3/jemalloc_ctl/stats/struct.allocated_mib.html</a></p>



<a name="232914794"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914794" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914794">(Apr 02 2021 at 15:51)</a>:</h4>
<p>nice!</p>



<a name="232914915"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914915" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914915">(Apr 02 2021 at 15:52)</a>:</h4>
<p>Hmm! wish I had known about (slash bothered looking for) this before the shrink-mem sprint!</p>



<a name="232914956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232914956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232914956">(Apr 02 2021 at 15:53)</a>:</h4>
<p>I don't know if it, well, works</p>



<a name="232915002"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232915002" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232915002">(Apr 02 2021 at 15:53)</a>:</h4>
<p>but seems plausible :)</p>



<a name="232916253"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916253" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916253">(Apr 02 2021 at 16:03)</a>:</h4>
<p>heaptrack appears to work with rustc too AFAICT, but is not able to extract the allocation sizes (6MB peak is _very_ suspicious). Some interesting results though:</p>
<div class="codehilite"><pre><span></span><code>$ heaptrack_print heaptrack.rustc.28607.zst | tail -n5
calls to allocation functions: 22089 (69028/s)
temporary memory allocations: 2314 (7231/s)
peak heap memory consumption: 5.66MB
peak RSS (including heaptrack overhead): 106.91MB
total memory leaked: 320.50KB
$ heaptrack_print heaptrack.rustc.28653.zst | tail -n5
calls to allocation functions: 22089 (68813/s)
temporary memory allocations: 2355 (7336/s)
peak heap memory consumption: 5.69MB
peak RSS (including heaptrack overhead): 106.25MB
total memory leaked: 320.50KB
$ heaptrack_print heaptrack.rustc.28755.zst | tail -n5
calls to allocation functions: 22085 (69231/s)
temporary memory allocations: 2298 (7203/s)
peak heap memory consumption: 5.68MB
peak RSS (including heaptrack overhead): 109.49MB
total memory leaked: 320.50KB
$ heaptrack_print heaptrack.rustc.29012.zst | tail -n5
calls to allocation functions: 22086 (69018/s)
temporary memory allocations: 2322 (7256/s)
peak heap memory consumption: 5.68MB
peak RSS (including heaptrack overhead): 109.66MB
total memory leaked: 320.50KB
</code></pre></div>
<p>Looks like number of allocations is still slightly not reproducible, even with <code>-Ccodegen-units=1</code>. The number of allocation function calls between runs differing is also somewhat curious.</p>



<a name="232916532"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916532" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916532">(Apr 02 2021 at 16:05)</a>:</h4>
<p>hmm I have not been able to get heaptrack to work with jemalloc - are you using the system allocator?</p>



<a name="232916549"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916549" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916549">(Apr 02 2021 at 16:05)</a>:</h4>
<p><a href="https://bugs.kde.org/show_bug.cgi?id=431746">https://bugs.kde.org/show_bug.cgi?id=431746</a></p>



<a name="232916576"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916576" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916576">(Apr 02 2021 at 16:05)</a>:</h4>
<p>I am not, I bet its just counting whatever other allocations happen to go through the system allocator.</p>



<a name="232916598"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916598" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916598">(Apr 02 2021 at 16:05)</a>:</h4>
<p>very strange, for me it hangs indefinitely</p>



<a name="232916603"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916603" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916603">(Apr 02 2021 at 16:05)</a>:</h4>
<p>but I did see things like <code>RawVec</code> in stack traces so <span aria-label="shrug" class="emoji emoji-1f937" role="img" title="shrug">:shrug:</span></p>



<a name="232916611"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916611" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916611">(Apr 02 2021 at 16:06)</a>:</h4>
<p>what version of heaptrack do you have?</p>



<a name="232916651"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916651" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916651">(Apr 02 2021 at 16:06)</a>:</h4>
<p>this would be super useful for me locally</p>



<a name="232916672"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916672" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916672">(Apr 02 2021 at 16:06)</a>:</h4>
<p>1.2.0, from nixpkgs</p>



<a name="232916692"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916692" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916692">(Apr 02 2021 at 16:06)</a>:</h4>
<p>I have 1.1.80, maybe that's the issue?</p>



<a name="232916925"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232916925" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232916925">(Apr 02 2021 at 16:08)</a>:</h4>
<p>hm, it might be system allocator, <code>config.toml</code> says <code>jemalloc=false</code> commented out, so false could be the default (and I bet it is).</p>



<a name="232917033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232917033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232917033">(Apr 02 2021 at 16:09)</a>:</h4>
<p>FWIW the heaptrack maintainer seemed to think it was a fundamental issue with static linking, I would be surprised for it to be fixed so quickly</p>



<a name="232917738"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232917738" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232917738">(Apr 02 2021 at 16:15)</a>:</h4>
<p>Jemalloc is not the default locally</p>



<a name="232918419"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/232918419" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#232918419">(Apr 02 2021 at 16:20)</a>:</h4>
<p>ah and we sadly cannot replace our various usual outputs with information we grab from jemalloc directly because many builds don't use jemalloc in the first place.</p>



<a name="233078144"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/reproducible%20memory%20usage%3F/near/233078144" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/reproducible.20memory.20usage.3F.html#233078144">(Apr 04 2021 at 12:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/reproducible.20memory.20usage.3F/near/232838174">said</a>:</p>
<blockquote>
<p>Hey <span class="user-mention silent" data-user-id="119009">eddyb</span>, would the <em>extensive</em> work you did on reproducible instruction counts also make it possible to get reproducible memory usage, to make statistics like max-rss more useful and more usable? Right now they fluctuate too much to detect small changes in memory usage.</p>
</blockquote>
<p>how close is this to landing btw? I don't see a way to see the instructions diff per-query in perf.rlo</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>